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OBJECT EXTRACTION 
IN IMAGES 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

This invention is directed to computer image processing, and more particularly, to 
object recognition and extraction in computer images. 

2. Background 

Image processing takes images from the real world-captured by cameras, infra-red 
sensors, ultrasound scanners, or other devices-and manipulates those images to achieve a 
desired result. Image processing is used in many applications, including the remote sensing 
of images beamed back from satellites, machine vision of faulty parts on a production line, 
and the enhancement and analysis of scanned images of the body. Recently, image 
processing has been incorporated into, a range of more mainstream computing applications, 
such as desktop publishing and multimedia. Multimedia techniques such as video image 
compression, enhancement, and warping all require image processing techniques. 

The objective of image processing is to visually enhance or statistically evaluate 
some aspect of an image not readily apparent in its original form. To that end, object 
recognition or object extraction becomes extremely important. Object recognition is a 
system's attempt to find an object within a particular image. The system rids the image of 
all elements that do not fit the system's definition of an object, compares the remaining 
image to the system's definition, and decides whether to recognize the remainder of the 
image as an object. Thus, automated object recognition consists of two tasks: (1) extracting 
relevant information from an image; and (2) making decisions about that information. 

In multimedia applications, object extraction is extremely important in indexing and 
accessing images quickly. For instance, news agencies frequently store footage of news 
stories in video databases. In order to properly index the images., the agency must have 
knowledge of the subjects contained in those images. One method for indexing the images 
would be to manually search the footage and divide it according to subject matter. 
Unfortunately, such a method would be very tedious and time-consuming. A faster approach 

1 
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would involve the use of image processing, A user would select a particular object and then 
retrieve all images containing that object. 

Despite its advantages, however, object extraction is extremely difficult. Most image 
processing works with gray-scale images. Thus, a system begins with a two-dimensional 
5 array of pixels, each of which has an integer value representing gray scale, or brightness. 
From there, the image processor can only discard information, it cannot meaningfully add 
information to what it has received. The complexities of image processing arise in 
determining which information is and is not relevant and how to retain the data important to 
making a decision about a given object. Moreover, the overlapping of objects makes 
10 multiple object extraction difficult as well. 

Several techniques have been developed to handle the complexities of object 
extraction. The two most common methods used in object recognition and extraction are: (1) 
region growing; and (2) edge detection. Region growing is an iterative bottom-up approach 
to object recognition. During region growing, an initial seed pixel is chosen to be the base 
15 region where the iteration will commence. Thereafter, for each of the eight pixels 

surrounding the seed pixel, the image processor decides whether to include the pixel with the 
seed pixel as part of the object. This process is repeated for every pixel within the image. 
Region growing is a limited process, however. Very often, when dealing with small regions, 
the region growing algorithm mistakenly attributes neighboring pixels to different objects. 
20 Thus, the algorithm fails to extract the proper object from the image. 

An alternative approach to region growing is edge detection. Edge detection is a top- 
down method for extracting objects. During edge detection, an image processor extracts an 
object by discerning significant changes between the boundaries of various objects. The 
processor examines factors such as the intensity value of a pixel and compares that value to a 
25 neighboring pixel. Edge detection poses additional problems, however, such as the failure to 
distinguish between different portions of the object. This failure results in non-object items 
being extracted along with the desired object. 

Moreover, both region growing and edge detection lack sufficient user input. With 
both methods, the user selects an image and then allows an algorithm to extract a desired 
30 object. During processing, the user cannot assist in the extraction of the object or offer any 
other meaningful input. In many instances, the user may be able to quickly assist in the 
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extraction process by specifying particular portions of an object within an image and/or 
eliminating certain undesired portions of an extracted object. 

SUMMARY OF THE INVENTION 

5 The present invention overcomes the problems of the prior art by incorporating 

region growing, edge detection, and a new containment process. The containment process 
tags particular pixels from a segmented object image. An output component then creates a 
new image using the tagged pixels from the original image. The containment processes 
limits the effects of holes within the extracted object. In addition, the present invention 

10 allows the user to assist in the extraction process. 

In accordance with the purpose of the invention, as embodied and broadly described 
herein the invention is software for extracting an object from an input image. The software 
receives seed coordinates of the object within the input image from a user. The system then 
creates an edge-detected image from the input image using edge detection techniques. The 

15 input image is then converted into gray scale data. In addition, data from the edge-detected 
image is converted into edge-detected gray scale data. The software then binarizes the edge- 
detected gray scale data.. Next, the software creates a segmented object image using the 
seed coordinates, the gray scale data from the input image, and the binarized edge-detected 
gray scale data. The segmented object image is then mapped in four directions to determine 

20 whether a pixel belongs to an outline of the object or to a background element. The pixels 
belonging to the outline of the object are stored. Finally, an output image is created from the 
input image based on the location of the pixels belonging to the outline of the object. 

In further accordance with the purpose of this invention, as embodied and broadly 
described herein the invention is a computer system having a memory comprising: a seed 

25 coordinate receiving component that receives seed coordinates of the object; an edge 
detection component that detects an edge of the input image; an image binarization 
component that binarizes the edge-detected image; a threshold computation component that 
computes a threshold for a region growing component: a region growing component that 
creates a segmented object image using the seed coordinates, data from the edge-detected 

30 image and data from the input image; a containment component that maps the segmented 
object image in at least one direction to determine whether a pixel belongs to an outline of 
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the object or to a background element and stores a location of the pixels belonging to the 
outline of the object; and an output component that creates an output image from the input 
image based upon the pixels determined to belong to the outline of the object. 

Objects and advantages of the invention will be set forth in part in the description 
5 which follows and in part will be obvious from the description or may be learned by practice 
of the invention. The objects and advantages of the invention will be realized and attained 
by means of the elements and combinations particularly pointed out in the appended claims. 

BRIEF DESCRIPTION OF THE DRAWINGS 

10 FIGURE 1 is a block diagram of a computer system, in accordance with a preferred 

embodiment of the present invention. 

FIGURE 2 is a block diagram of the components of the image processor of the 
present invention. 

FIGURES 3A & 3B are a flow chart showing steps performed by the main 
1 5 component of the image processor of FIGURE 1 . 

FIGURE 4 is an illustration of a 3 X 3 group of pixels in an input image. 
FIGURE 5 is a flow chart describing steps performed by the edge detector of 
FIGURE 2. 

FIGURE 6 is a flow chart describing steps performed by an image input subroutine. 
20 FIGURE 7 is a flow chart illustrating steps performed within a threshold computation 

routine. 

FIGURE 8 is a flow chart illustrating steps performed by region grower 
FIGURE 9 is a flow chart illustrating steps performed by the containment 
component. 

25 FIGURE 10 is a flow chart illustrating steps performed within the image output 

routine. 

FIGURES 11a and 1 lb are illustrations of an image containing an object to be 
extracted. 
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DETAILED DESCRIPTION 

Reference will now be made in detail to the invention, examples of which are 
illustrated in the accompanying drawings. Wherever possible, the same reference numbers 
will be used throughout the drawings to refer to the same or like parts. 
5 The present invention is an apparatus and method for extracting an object from an 

image. Although the present invention is illustrated as software, it may also be implemented 
as a hardware component in, for example, a graphics card in a computer system. FIGURE 1 
is a block diagram of a computer system 100, in accordance with the present invention. 
Computer system 100 includes a CPU 102, a memory 104, and input/output lines 106; an 
10 input device 150, such as a keyboard or mouse; and a display device 160, such as a display 
terminal. Computer system 100 can also include numerous elements not shown for the sake 
of clarity, such as disk drives, network connections, additional memory, additional CPUs, 
etc. 

Memory 104 includes a source image file 1 10 containing an object to be extracted, an 

15 image processor 111 for extracting the object, and a target image file 124 that will contain 
the extracted object. FIGURES 11a and 1 lb are examples of an image 1 100 that may be 
contained in image source file 1 10. The image 100 includes a beaver object 1 105 and a tree 
object 1110. For the purposes of this discussion, the user desires to extract the beaver object 
1 105 from the image 1 100. FIGURE 1 lb shows the extracted beaver object. The image 

20 processor 1 1 1 is preferably a Unix command-line executable C programming language 
program, containing several components. It should be apparent that other programming 
languages may be used. 

FIGURE 2 is a block diagram of image processor 111. Image processor 1 1 1 contains 
several components, including an image file converter component 1 12, an image input 

25 component 124, an edge detector component 1 14, a seeded region grower component 1 16, a 
threshold computation component 1 17, a containment component 118, and an image output 
component 126. Each component of Image processor 1 1 1 contains sub-components that will 
be described in greater detail below. Image processor 1 1 1 includes sub-components, as well, 
such as a main routine. Image processor 1 1 1 inputs source image file 1 10 containing the 

30 desired object, extracts the object from the image source file 110 and outputs the extracted 
object into target image file 124. All parts of image processor 111, preferably, are embodied 
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as instructions stored in memory 104 and performed by CPU 102 although these components 
may also be embodied in hardware. Image processor 1 1 1 may be embodied as instructions 
stored on removable storage medium 92 which is read by drive 90. Memory 104 also 
contains additional information, such as application programs, operating systems, data, etc., 
5 which are not shown in the figure for the sake of clarity. A preferred embodiment of the 
invention runs under the Solaris operating system, Version 2.3 and higher. Solaris is a 
registered trademark of Sun Microsystems, Inc. Other operating systems and/or 
environments may be used as well. 

As stated above, image processor 1 1 1 is preferably a command-line-based executable 
10 application. As such the application may accept arguments from the command line. The 
image processor 1 1 1 accepts the following program arguments from a user at the command 
line: (1) "-x (X)(Y)", where X and Y are seed coordinates of the desired object within 
source image file 1 10; (2) "-c (R)(G)(B)'\ where the user may assign the RGB (red, green, 
blue) value of the background of the target image file (the image processor 1 11, preferably, 
15 defaults to a background color, such as white, where (R)=(G)=(B)=255); and (3) "-a (0/1)", 
where "0" is chosen if the user desires the extraction of a single object and "1" is chosen if 
the user desires the extraction of multiple objects. 

All C programming language programs begin in a "mam" subroutine. The main 
subroutine is then responsible for calling other subroutines in the program. FIGURE 3 is a 
20 flow chart showing steps performed within the main subroutine of image processor 111. The 
main subroutine of image processor 1 1 1 is responsible for accepting the command-line 
program arguments from the user as described above, performing various checks to ensure 
the validity of those program arguments, and calling other subroutines and components of 
image processor 1 1 1 to perform input of the source image file 110, extraction of the desired 
25 object, and output of the target file 124 containing the extracted image. In a first step 302, 
image processor 1 1 1 reads in the user-entered command-line program arguments (including 
the seed coordinates) and stores these arguments in memory 104. Next, in step 304, image 
processor 1 1 1 verifies that the command line arguments are valid arguments by comparing 
the arguments to a list of stored arguments. If the arguments are not valid, the image 
30 processor immediately halts execution in step 306. The image processor 1 1 1 may also 
provide an indication of invalidity to the user by, for example, displaying "Error: Invalid 
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Input Parameters" on display device 1 60. If the arguments are valid" processing continues at 
step 308. In step 308, the image processor 1 1 1 allocates a block of memory for a plurality of 
dynamic arrays that will be used by the various components of the image processor 111. 
Specifically, image processor 1 1 1 creates a two-dimensional grayjarray array, a two- 
5 dimensional color _array array, a header array array, a boundary jxrray array. The main 
subroutine then calls image file converter component 1 12 in step 312. 

Image file converter component 1 12 converts images from one file format to another. 
Among the various image file formats that may be converted are joint photographic experts 
group ("JPEG") format, graphic image file ("GIF") format, and bitmap ("BMP") format, 

10 although other formats may be converted. Image file converter 1 12 converts input source 
image file 1 10 from its original format to a portable pixmap ("PPM") format. Image file 
converter 1 12 then stores the converted input image in a temporary intermediate PPM image 
file 115. Image conversion is performed using techniques known in the art, such as those 
used in the software sold under the trademark "Image Magic" by Eastman Kodak Company. 

15 Other image conversion tools may also be used to convert input image source file 1 10 to 
PPM format. 

Once image converter 1 1 2 has converted the image source file 1 10 to the PPM 
format, image converter 1 12 returns an intermediate PPM image 1 15 to the main subroutine. 
In step 316, the main subroutine edge-detects the image by calling the edge detector 

20 component 114. Edge detector component 114 detects the edge of input source image 110. 
Both the conversion of the image source file to PPM format and the edge-detection of the 
source file 110 may be performed in a parallel manner to improve processing speed. Edge 
detection is performed using a Laplacian method. In the Laplacian method, an examined 
pixel is considered to be the center of a 3 X 3 window of pixels. FIGURE 4 is an illustration 

25 of a 3 X 3 window 400 of pixels that create an image. The examined pixel 401 lies in the 

center of the window 400. Since the local variance is larger near the boundary regions of the 
images, the calculated variance over the tested window 400 may be used to determine the 
resolution at a given pixel. Based on the determined local variance, a center test pixel 401 
with a similar intensity value to those its neighboring pixels in the window 400 is determined 

30 to be homogeneous at that resolution. Since the noise in the uniform region is more 
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objectionable than in the boundary region, the noise elements in the uniform region must be 
suppressed or smoothed. 

FIGURE 5 is a flow chart describing the edge detection process performed by the 
edge detector component 114. In step 502, edge detector component 1 14 enters a double 
nested loop to edge-detect every pixel contained in the image. The double nested loop 
allows the edge detector component 1 14 to traverse horizontal and vertical coordinates 
within an image. In step 506, within the loops, the local mean is estimated. Next, in step 
508, the local variance is calculated over the window 300. The local variance is assumed to 
have an expected value that is smaller than the peak of its distribution calculated over an 
entire image, where the distribution is assumed to be unimodal. Each pixel is then assigned 
an integer value according to the comparison results. The integer value represents a 
threshold level used to estimate the presence of an edge. Each time a test is conducted to 
detect the presence of an edge, the average mean of that pixel in relation to its neighbors is 
computed and compared to this threshold The edge detector 1 14 repeats this process for 
every pixel in the image. When all of the pixels have been assigned an integer value, edge 
detector 1 14 creates an intermediate edge-detected image 119. The edge detector 1 14 returns 
the edge-detected image 1 19 to the main subroutine. The edge detection process may be 
performed by an edge detection component of the Image Magic software described above. 
The edge-detected image is converted to PPM format to create an edge-detected PPM image 
file. FIGURE 10 illustrates the results of the edge detection on the image 900. 

Once control returns to main, the main subroutine then passes control to the image 
input component 121 in step 320. The image input component 121 performs several tasks. 
First, the component 121 stores the intermediate input PPM image 115 created by image file 
converter 1 12. Second, the header of the PPM image 1 15 is stored in an intermediate 
location (an array) and the image information stored in the header, including the image 
dimension, is extracted. Third, the raw RGB data is extracted from the remainder of the 
original PPM image. This raw data will be passed to several components for computation. 
Fourth, the image input component 121 converts the RGB color PPM image 1 1 5 into gray 
scale and stores the gray scale data into another array. Finally, the raw RGB data extracted 
from the original PPM image is passed to the edge detection component, where it is 
converted into gray scale data and binarized. 
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FIGURE 6 is a flow chart describing steps performed during the input image routine 
performed by image input component 121. In step 602, the routine opens input PPM image 
1 15 and edge-detected PPM image 1 19 for reading. In step 604, the input component 125 
reads the header of input PPM image 115 and edge-detected PPM image 119. In step 606, 
5 the input component 125 stores the read headers of both files into the header jxrray array. 

Next, the image input component 125 enters two nested loops. The initialization 
statement of the first "outside" loop sets a loop control variable, /, equal to zero. The 
expression of the first loop (i.e., the conditional expression that determines whether or not 
the loop will repeat) requires the loop control variable, /, to be less than or equal to X. where 

10 X\s the X -coordinate parameter of the original input image. Following each execution of the 
loop, the loop control variable is incremented by a value of 1 . The initialization statement of 
the second "inside" loop sets a second loop control variable equal to zero. The expression of 
the second loop requires the second loop control variable, J ? to be less than or equal to Y, 
where Y is the Y-coordinate parameter of the original input image. Within the loops, in step 

15 610, the input routine reads red pixel values, green pixel values, and blue pixel values at the 
particular X and Y coordinate in the input PPM image 115. In step 612, the routine stores 
the RGB color pixel value in the array, color jxrray (I, J), where I & J are the loop control 
variables for the first and second loops, respectively. Thus, I and J correspond to a pixel at a 
particular coordinate. Next, the input routine changes the RGB color pixel to a gray scale 

20 pixel using an image processing function. For example, the following function would 
provide a gray level value of an RGB pixel of 24 bits having a red component, a green 
component, and a blue component: 

Gray_Level [R, G, B] = .299 * Color_image[R] + .587 * Color_image[G] -.114* 
ColorJmage[B]. 

25 If Gray_Level[R, G, B] < 0, then Gray_Level[R. G, B] =0 

Else Gray_Level[R, G, B] = the floor function of Gray_Level[R, G, B] +.5 

In step 616, the component 125 stores the gray pixel value in the location gray array (I,J). 
In step 620, the component 125 reads the RGB values of the color edge-detected image. 
30 Next, the input component 125. in step 622 converts the color edge-detected image into gray 
scale data. Next, in step 624, the component 121 binarizes the edge-detected gray scale 
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pixel. Specifically, if the gray scale value of the edge-detected pixel is greater than 100, the 
pixel is given a binary value of 1 . If the gray scale value of the edge-detected pixel is less 
than 100, the pixel is given a value of 0. The routine then stores the binarized pixel values 
from the edge-detected image in the location boundary jarr ay (I, J). The binarization makes it 
easier to use the edge detection image in boundary recognition. The second loop is then 
closed followed by the first loop. In step 626, the input component 121 closes all of the 
opened files. The loops are shown in their entirety below: 
For (1=0; I<X; I++) { 

For (J=0; J<Y;J++) { 

Read Red Pixel value of input image file; 

Read Green pixel value of input image file; 

Read blue pixel value of input image file; 

Store RGB color pixel value in the color array (I, J); 

Convert to gray scale pixel (see above formula); 

Store gray pixel value in the location gray_array(I t J)\ 

Read Red Pixel value of edge-detected image file 

Read Green Pixel value of edge-detected image file 

Read Blue Pixel value of edge-detected image file 

Change to gray scale pixel; 

Binarize edge-detected gray scale pixel; 

Store gray binarized pixel value in the location boundary _array(lj)\ 

} 

} 

Close all files 

Next, the main routine, in step 324, calls threshold computation component 1 17. 
Threshold computation component 1 1 7 computes a threshold value that will be used by the 
seeded region growing component 116 during the seeded region growing of the image. 
Region growing is an iterative approach, wherein a decision is made regarding the inclusion 
or exclusion of each pixel in the image. An initial seeded pixel centered in a 3 X 3 region of 
pixels is chosen to be the base region where the iteration will commence. Thereafter, the 
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software decides whether each of the 8 connected neighboring test pixels in the current 
iteration will be either combined with the base region or excluded from the base region. 
Each test pixel is examined from two perspectives: (1) a boundary examination to determine 
whether a boundary exists in each pixel; and (2) a homogeneity examination to determine if 
5 the pixel is similar to the base region. If both of these examinations have succeeded, the 
tested pixel is merged with the base region. "Homogeneity" represents the level of color or 
light intensity in the region. If the absolute difference of the mean of the test pixel value is 
within a pre-specified "threshold" of the mean of the base region, the test pixel is considered 
to be part of the desired object and, therefore, part of the base region. 

10 FIGURE 7 is a flow chart illustrating how the threshold computation component 117 

determines the threshold. In particular, the component 117 computes the variance of the 
input image and multiplies the variance by a predetermined constant variable, M. M is a 
constant value, where 0 < M < 1 . M is preferably derived from simulations on a variety of 
training images. The value of M affects the degree of object segmentation and, therefore, the 

15 quality of the segmented object. At the outset of the routine, in step 702, the variables Mean, 
Variance, and Temporary, are all set equal to 0. The loop proceeds in steps 706-708 as 
follows: 

For (1=0; I<X dimension; I-H-) { 

For (J=0; J<Y dimension; J++) { 
20 Mean = Mean + gray_array(I,J) 

Tempora?y = Temporary + gray jxrr ay (l J) * gray array (I J) ; 

} 

} 

25 The value stored in the array, gray array is derived from the image input routine 

described above. Specifically, the array stores the gray scale value of the pixel. From the 
foregoing loop, the variable Mean is equal to the sum of the array, grayjarray. A temporary 
variable is used in the calculation of the variance. The actual mean is then calculated in step 
710 by dividing by the number of array elements: 

30 Mean = Mean/(X dimension * Y dimension) 
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Similarly, in step 712, the temporary variable is divided by the total number of array 
elements: 

Temporary = Temporary/(X dimension * Y dimension) 
Next, in step 714, the variance is calculated as follows: 

5 Variance = ^(Temporary - Mean * Mean) 

Finally, in step 716, the threshold is determined: 
Threshold = M * Variance 

Once the threshold has been computed, the main routine passes control to region 
grower component 116. Region grower performs a recursive subroutine responsible for the 

10 task of extracting the desired object. The seed region is initially supplied to the component 
when the extraction process begins. At the beginning of the segmentation process, the user 
specifies the seed region by, for example, clicking on a position within the object to be 
segmented. The program translates the mouse click to aX-seed and Y-seed coordinates. 
This translation may be performed using standard scripting languages. The region grower 

15 116 performs a recursive check on neighboring regions or pixels for homogeneity. If the 

neighboring regions are homogeneous, they are added to the grown region. The boundary of 
the neighboring region is also checked for object boundary. Since the process is recursive, 
the region grower 116 includes a tagging mechanism to ensure that processed pixels are not 
re-processed. If the absolute difference of the mean of a tested pixel is within the pre- 

20 determined threshold of the mean of the value of the base region pixels, the tested pixel is 
considered to be part of the object and is added to the base region. 

FIGURE 8 is a flow chart illustrating steps performed by region grower 116. In step 
802, the region grower 1 16 initializes the elements of an array, Tag (I J), to zero. The array, 
Tag (I,J)> is used to ensure that processed pixels are not reprocessed within the recursive 

25 subroutine of the region grower 116. The array elements are initialized within two loops. 
The first loop that executes while the loop control variable, I, is less than or equal to the X 
dimension of the image. The second loop executes while the second loop control variable, J, 
is less than or equal to the Y dimension of the image. The loop proceeds, as follows: 
For (I = 0; I<X dimension; I++) { 

30 For (J = 0; J<Y dimension; J++) { 

Tag (I,J) = 0; 
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} 

} 

Next, in step 804, the region grower 116 sets a variable, Total_Mean equal to the 

5 value of the array element, gray_array (Xseed, Yseed), where Xseed and Yseed are the seed 
coordinates of the desired image, provided by the user at start-up. The array, grayjurray 
was created by the image input subroutine, and contains the gray pixel values of the input 
image. In addition, region grower 1 16 sets a variable Count equal to 1. 

The region grower 116 then enters a recursive subroutine. Segment, that performs the 

10 extraction of the image. The initial parameters to the segment routine are Xseed, Yseed, 
TotalMean, and Count. Total_Mean divided by Count represents the average mean of the 
grown region. This will be compared to the value of the pixels under examination. Thus, 
extraction begins at the seed coordinates of the object. Thereafter, the routine calls itself 
with different coordinates. The routine executes only if the elements of boundary jarr ay 

1 5 (Xseed, Yseed), and Tag (Xseed, Yseed) are equal to zero. A value of one in the binarized 
edge-detected image would indicate that a boundary exists. Thus, the pixel is considered if 
its value is equal to zero. The array, boundary jar ray, was created by the image input 
subroutine and contains the binarized gray scale pixel of the edge-detected image. Next, 
region grower 116 sets the variable Mean equal to grayjarray (Xseed, Yseed) and the 

20 variable Area_Mean is then set to Total_Mean/Count. A variable, Difference, is set equal to 
the absolute value of AreaJMean - Mean. Next, the region grower 116 determines whether 
the calculated Difference is less than the value of the Threshold as computed by the 
threshold computation component 117. If Difference is less than Threshold, the region 
grower 1 16 extracts the object by setting the pixel values of an array, 

25 segmented_object_array (Xseed, Yseed), equal to 1. The value of Total Mean is 

incremented by the value of Mean. The region grower 116 increments the Count variable 
and the tag of the subject pixel is set equal to 1. By setting the Tag elements equal to 1, the 
pixels will not be reprocessed. The region grower 116 then recursively calls itself to process 
each pixel within the desired object. The subroutine. Segment, is shown in its entirety 

30 below: 

Segment (Xseed, Yseed, Total_Mean, Count) { 
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if (boundary_array (Xseed, Yseed) =0) and (Tag (Xseed, Yseed) = 0) 
{ 

Mean = gray_array (Xseed, Yseed); 
Area_Mean = Total_Mean/Count; 
Difference = abs(Area_Mean-Mean); 
If (Difference < Threshold) 

{ 

segmented_object_array (Xseed, Yseed) = 1 ; 
Total_Mean = Total Mean + Mean; 
Count++; 

Tag (Xseed, Yseed) = 1 ; 



if (Xseed > 0) 

{ Segment (Xseed - 1, Yseed, Total_Mean, Count);} 
if (Xseed < X dimension) 

{ Segment (Xseed + 1, Yseed, Total_Mean, Count);} 
if ( Yseed >0) 

{ Segment (Xseed, Yseed - 1, Total_Mean, Count);} 



if(Yseed<0) 

{ Segment (Xseed, Yseed + 1, Total_Mean, Count);} 

} 

} 

} 

Once the pixels of the desired object have been extracted and stored in an array, main 
calls containment component 1 18 in step 328. Containment component 118 maps a 
segmented binary object image from four different directions. The segmented binary object 
image is an image resulting from assigning a binary value to each pixel in the image, where a 



14 



WO 01/08098 



PCT/US00/19900 



value of "1" indicates that the pixel is part of the desired object and a value of "0" indicates 
that the pixel is not part of the desired object. The object of the containment component is to 
tag each pixel of the extracted object as being either an outline of the object of the 
background image. The containment component outputs an array containing tagged pixels 
5 which will be used to construct the final extracted image. FIGURE 9 is a flow chart 
illustrating steps performed by the containment component 118. 

In step 902, containment component 1 1 8 begins mapping in a first direction from X = 
0 to X = X dimension. In particular, mapping is performed as follows: the component 1 1 8 
sets X = to 0. Next, while X is less than or equal to the X dimension of the image, the pixel 

10 is tagged as part of the background if the pixel value is equal to zero. Otherwise, the 
boundary reached is set to true. The value of X is then incremented. In step 910, the 
containment component begins mapping in the reverse direction from X=X dimension to X 
= 0. X is initially set to X dimension. The mapping then proceeds until X = 0. Mapping in 
the Y direction is similar. Once the mapping is complete, the containment component has 

15 obtained the boundary coordinates of the object. Those coordinates are then used to replace 
the extracted object with the object from the original input image. 

Following the mapping, main passes control to the image output component 126. 
The image output component 126 stores the output object image array into a temporary 
intermediate output PPM image. The image will be later converted into the appropriate 

20 format. In addition, the image output routine distinguishes between the extracted object 
pixels and background pixels. Background pixels are those taken either from that of a 
default value (white), a uniform background color supplied by the user, or the last extracted 
image pixels. 

FIGURE 10 is a flow chart illustrating steps performed by the image output 
25 component 126. In step 1002. image output component 126 opens an output PPM file for 
writing. In step 1004, the component 126 opens a multiple extraction file for writing. In 
step 1006, the component 126 stores the headers from the PPM file from the header array. 
The component 126 then enters a loop to store RGB color _arr ay in the temporary 
intermediate output PPM file opened in step 802. The loop is shown below: 
30 For (I = 0; I<X dimension; I++) { 

For (J = 0; J < Y dimension; J++) { 



15 



WO 01/08098 PCT/US00/19900 

If (segmented_object_array (I,J) is not tagged) { 

Store RGB color_Array (I,J) in the temporary intermediate 

output Ppm; 

Store RGB color_array (I,J) in the temporary multiple extract 

5 file} 

else { 

Store RGB bg_array(I,J) in the temporary intermediate output 

PPM; 

Store RGB bg__array (I,J) in the temporary multiple extraction 

10 file} 

} 

} 

The image output component 126 then closes all of the files. 

Having thus described a preferred embodiment of an image processor, it should be 
15 apparent to those skilled in the art that certain advantages have been achieved. It should also 
be appreciated that various modifications, adaptations, and alternative embodiments thereof 
may be made within the scope and spirit of the present invention. The invention is further 
defined by the following claims: 
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CLAIMS 

What is Claimed is: 

1. A method, performed by a computer system, for extracting an object from an 
input image comprising the steps of: 

receiving seed coordinates of the object within the input image from a user; 

creating an edge-detected image from the input image; 

converting data from the input image into gray scale data; 

converting data from the edge-detected image into edge-detected gray scale data; 

binarizing the edge-detected gray scale data; 

creating a segmented object image using the seed coordinates, the gray scale data 
from the input image, and the binarized edge-detected gray scale data; 

mapping the segmented object image in at least one direction to determine whether a 
pixel belongs to an outline of the object or to a background element; 

storing a location of the pixels belonging to the outline of the object; and 

creating an output image from the input image based on the location of the pixels 
belonging to the outline of the object. 

2. The method for extracting an object from an input image, as recited in Claim 1, 
wherein the step of creating an edge-detected image further comprises the steps of: 

selecting a pixel within the input image; 

comparing a resolution of the selected pixel to a resolution of a plurality of adjacent 
pixels; and 

assigning a value to the selected pixel based on the comparison. 

3. The method for extracting an object from an input image, as recited in Claim L 
wherein the step of creating a segmented image further comprises the steps of: 
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creating a base region beginning with a pixel at the seed coordinates; 
analyzing additional pixels for homogeneity with neighboring pixels; and 
growing the base region by adding homogeneous pixels to the base region. 

4. The method for extracting an object from an input image, as recited in Claim 3, 
wherein the step of creating a segmented image further comprises the steps of: 

computing a variance of the input image; and 

multiplying the variance by a constant variable to determine a threshold value; 
wherein a pixel is added to the base region if an absolute difference of a mean of the 
value of the pixel is less than the threshold value. 

5. The method for extracting an object from an input image, as recited in Claim 1, 
wherein the step of mapping the segmented object image further comprises the steps of: 

mapping the segmented object image in a first direction from a pixel having an X 
coordinate equal to zero to a pixel having an X coordinate equal to the greatest X coordinate 
in the segmented object image; and 

mapping the segmented object image in a second direction from a pixel having an X 
coordinate equal to the greatest X coordinate in the segmented object image to a pixel having 
an X coordinate equal to zero. 

6. The method for extracting an object from a computer-generated input image, as 
recited in Claim 5, wherein the step of mapping the segmented object image further 
comprises the steps of 

mapping the segmented object image in a third direction from a pixel having a Y 
coordinate equal to zero to a pixel having a Y coordinate equal to the greatest X coordinate 
in the segmented object image; and 

mapping the segmented object image in a fourth direction from a pixel having a Y 
coordinate equal to the greatest Y coordinate in the segmented object image to a pixel having 
a Y coordinate equal to zero. 
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7. The method for extracting an object from an input image, as recited in Claim 1 , 
further comprising the step of converting the input image to a portable pixmat file format. 

8. The method for extracting an object from an input image, as recited in Claim 7, 
further comprising the step of converting the output image to its original file format. 

9. A method, performed by a computer system, for extracting an object from an 
input image comprising the steps of: 

receiving seed coordinates of the object within the input image from a user; 
creating an edge-detected image from the input image; 

creating a segmented object image using the seed coordinates, data from the edge- 
detected image and data from the input image; 

mapping the segmented object image in at least one direction to determine whether a 
pixel belongs to an outline of the object or to a background element; 

storing a location of the pixels belonging to the outline of the object; and 

creating an output image from the input image based on the location of the pixels 
belonging to the outline of the object. 

10. The method for extracting an object from an input image, as recited in Claim 9, 
wherein the step of creating a segmented object image further comprises the steps of: 

converting data from the input image into gray scale data; 

converting data from the edge-detected image into edge-detected gray scale data; and 
binarizing the edge-detected gray scale data. 

1 1 . The method for extracting an object from an input image, as recited in Claim 10, 
wherein the step of creating a segmented object image further comprises the steps of: 

creating a base region beginning with a pixel at the seed coordinates; 
analyzing additional pixels for homogeneity with neighboring pixels; and 
growing the base region by adding homogeneous pixels to the base region. 



19 



WO 01/08098 



PCT/USOO/19900 



12. The method for extracting an object from an input image, as recited in Claim 9, 
wherein the step of mapping the segmented object image further comprises the steps of: 

mapping the segmented object image in a first direction from a pixel having an X 
coordinate equal to zero to a pixel having an X coordinate equal to the greatest X coordinate 
in the segmented object image; and 

mapping the segmented object image in a second direction from a pixel having an X 
coordinate equal to the greatest X coordinate in the segmented object image to a pixel having 
an X coordinate equal to zero. 

13. The method for extracting an object from an input image, as recited in Claim 12, 
wherein the step of mapping the segmented object image further comprises the steps of: 

mapping the segmented object image in a third direction from a pixel having a Y 
coordinate equal to zero to a pixel having a Y coordinate equal to the greatest X coordinate 
in the segmented object image; and 

mapping the segmented object image in a fourth direction from a pixel having a Y 
coordinate equal to the greatest Y coordinate in the segmented object image to a pixel having 
a Y coordinate equal to zero. 

14. The method for extracting an object from an input image, as recited in Claim 13, 
wherein the step of storing a location of the pixels belonging to the outline of the object 
further comprises the steps of: 

storing the location of the pixels along the first direction belonging to the outline of 
the object; 

storing the location of the pixels along the second direction belonging to the outline 
of the object; 

storing the location of the pixels along the third direction belonging to the outline of 
the object; 

storing the pixels along the fourth direction belonging to the outline of the object. 
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15. The method for extracting an object from an input image, as recited in Claim 9, 
wherein the step of creating an output image from the input image based on the location of 
the pixels belonging to the outline of the object further comprises the steps of: 

removing the pixels defined by the boundary of the stored locations from the input 

image; 

placing the removed pixels in the output image at the stored locations. 

16. The method for extracting an object from an input image, as recited in Claim 15, 
further comprising the steps of placing a plurality of default background pixels in the output 
image at locations other than the location of the object. 

17. A computer system for extracting an object from an input image, the computer 
system having a memory, the computer system comprising/ 

a seed coordinate receiving component that receives seed coordinates of the object; 

an edge detection component that detects an edge of the input image; 

a region growing component that creates a segmented object image using the seed 
coordinates, data from the edge-detected image and data from the input image; 

a containment component that maps the segmented object image in at least one 
direction to determine whether a pixel belongs to an outline of the object or to a background 
element and stores a location of the pixels belonging to the outline of the object; and 

an output component that creates an output image from the input image based upon 
the pixels determined to belong to the outline of the object. 

18. The computer system for extracting an object from an input image, as recited in 
Claim 17, further comprising an image converter that converts the input image into a 
different format. 

19. The computer system for extracting an object from an input image, as recited in 
Claim 17, further comprising a threshold computation component that computes a threshold 
value used by the region growing component. 
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20. The computer system for extracting an object from an input image, as recited in 
Claim 19, wherein the region growing component further comprises: 

means for creating a base region based upon the seed coordinates; 
means for recursively determining a homogeneity of a plurality of pixels with a 
neighboring pixel; 

means for adding homogeneous pixels to the base region to create a segmented 

object. 

21 . A computer program product, comprising: 

a computer usable medium having a computer readable program code mechanism 
embodied therein configured to receive seed coordinates of an object within an input image 
from a user; 

a computer readable program code mechanism configured to create an edge-detected 
image from the input image; 

a computer readable program code mechanism configured to create a segmented 
object image using the seed coordinates, data from the edge-detected image and data from 
the input image; 

a computer readable program code mechanism configured to map the segmented 
object image in at least one direction to determine whether a pixel belongs to an outline of 
the object or to a background element; 

a computer readable program code mechanism configured to store a location of the 
pixels belonging to the outline of the object; and 
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a computer readable program code mechanism configured to create an output image 
from the input image based on the location of the pixels belonging to the outline of the 
object. 

22. The computer program product of Claim 2 1 , wherein the computer readable 
program code mechanism in the program product further comprises: 

a computer readable program code mechanism configured to convert data from the 
input image into gray scale data; 

a computer readable program code mechanism configured to convert data from the 
edge-detected image into edge-detected gray scale data; and 

a computer readable program code mechanism configured to binarize the edge- 
detected gray scale data. 
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